Fast Sorted-Set Intersection using SIMD Instructions

In this paper, we focus on sorted-set intersection which is an important part in many algorithms, e.g., RID-list intersection, inverted indexes, and others. In contrast to traditional scalar sorted-set intersection algorithms that try to reduce the number of comparisons, we propose a parallel algorithm that relies on speculative execution of comparisons. In general, our algorithm requires more comparisons but less instructions than scalar algorithms that translates into a better overall speed. We achieve this by utilizing efficient single-instruction-multiple-data (SIMD) instructions that are available in many processors. We provide different sorted-set intersection algorithms for different integer data types. We propose versions that use uncompressed integer values as input and output as well as a version that uses a tailor-made data layout for even faster intersections. In our experiments, we achieve speedups up to 5.3x compared to popular fast scalar algorithms.